Real-Time Tracking of Articulated Human Models Using a 3D Shape-from-Silhouette Method

This paper describes a system which acquires 3D data and uses the data to track an eleven degree of freedom human model in real-time. Using four cameras we create a time-varying volumetric image (a visual hull) of whatever object is moving in the space observed by all four cameras. We are currently operating the sensor in a volume of approximately 500,000 voxels (1.5 inch cubes) at a rate of 25 Hz. We are able to track the upper body dynamics of a human within the workspace. The system is able to track the x, y position of the body, a torso rotation about the z axis, along with four rotations in each arm. Tracking occurs using a method we developed to be extremely fas~ which allows both data acquisition and tracking to occur on one computer at a rate of 16 Hz. We also developed a calibration procedure, which allows the system to be reconfigured or even moved and to quickly be recalibrated. Furthermore the system utilizes another computer to visualize either the voxel data overlaid with the joint locations or to view a human avatar, both of which are driven in real-time. Lastly our system has been implemented to perform crane gesture recognition, and has been linked with a large robotic arm to simulate crane movements. Corresponding author Jason Luck Dept. Of Engineering Systems Colorado School of Mines 1500 Illinios St. Golden, CO 80401 USA